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1 ABSTRACT 

This paper describes the Robotic 
Air Vehicle system sponsored by both the 
Defense Advanced Research Projects 
Agency (DARPA) and Air Force Wright 
Aeronautical Laboratories ( AFWAL ) . The 
program's objective is to design, 
implement, and demonstrate cooperating 
expert systems for piloting robotic air 
vehicles. The development of this 
system merges conventional programming 
used in passive navigation with 
Artificial Intelligence techniques such 
as voice recognition, spatial reasoning, 
and expert systems. The individual 
components of the RAV system are 
discussed as well as their interactions 
with each other and how they operate as 
a system. 


2 INTRODUCTION 

Challenging modern air defenses 
poses significant dangers such as loss 
of crew and aircraft. Intelligent 
unmanned flight systems can provide a 
viable solution to eliminate the loss of 
high-value aircraft and complement our 
manned force. The technology to allow 
the intelligence and adaptability of a 
pilot to be added to an unmanned flight 
system is being developed on the Robotic 
Air Vehicle (RAV) program. The RAV 
contract was awarded in September of 
1985 by the Defense Advanced Research 
Projects Agency (DARPA) and Air Force 
Wright Aeronautical Laboratories 
(AFWAL). The program's goals are to 
design, implement, and demonstrate 
cooperating expert systems for piloting 
robotic air vehicles. The approach 
being used by Texas Instruments is to 
combine conventional programming with 
Artificial Intelligence (AI) techniques. 
This approach leverages established 
technologies, such as 4 control theory and 
navigational terrain algorithms, with 
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more recent techniques such as expert 
systems that give the RAV system the 
ability to plan, execute, and alter its 
mission. The mission scenario addressed 
in this paper is the reconnaissance of 
heavily defended areas. 

The AI techniques applied in the 
RAV system are natural language 
understanding, voice recognition, expert 
systems, and spatial databases. The 
conventional software systems in the RAV 
system are the aircraft simulation, 
passive navigation, and the terrain 
following/ terrain avoidance route 
planner. 

The following section, Robotic Air 
Vehicle System Software Architecture, 
explains the individual systems, both AI 
and conventional, that compose the RAV 
system. The RAV Hardware section 
describes the types of computer systems 
and their configuration required to run 
the entire RAV system. The 
Reconnaissance Mission Example section 
illustrates the interaction of various 
RAV system components executing during a 
reconnaissance mission. 


3 ROBOTIC AIR VEHICLE SYSTEM SOFTWARE 
ARCHITECTURE 



Figure 1. RAV SYSTEM SOFTWARE 
ARCHITECTURE 


335 





3,1 Command Inputs 

The Robotic Air Vehicle system has 
two methods for command input: a 
natural language interface and a voice 
recognition system. During system 
execution, either input method can be 
used provided the user completes a 
command with one method before changing 
to the other method to enter the next 
command. 

A natural language interface is 
provided to the expert systems through 
Texas Instruments NLMenu (TM) . NLMenu 
is a menu-based approach to natural 
language understanding. The user is 
presented with a series of menus from 
which valid commands can be built. The 
menus are semantically constrained by a 
defined grammar so that only valid 
phrases within a context can be 
generated. The TI-DARPA fighter pilot 
grammar used in this interface is a set 
of in-flight commands used by pilots in 
general that has been tailored to the 
characteristics of high performance 
aircraft. This interface establishes an 
easy means to view the valid commands, 
to enter only complete commands, and to 
quickly enter commands for running the 
RAV system during knowledge engineering 
sessions with piloting experts. 

Voice recognition technology is 
applied to the RAV system to enable 
communication with air traffic control 
or other manned aircraft and to receive 
intel and weather updates. The same 
TI-DARPA fighter pilot grammar used for 
the NLMenu system is also used in the 
grammar for the Voice system. The voice 
recognition in the RAV system uses the 
connected word recognition technology 
developed under the DARPA Robust Speech 
contract. This technology is currently 
implemented on the Texas Instruments 
Odyssey signal processing board. 


3.2 Mission Plan 

A route planner generates the 
mission plan using a conventional 
algorithmic approach. Given both a 
starting and ending point, the route 
planner finds an optimal path between 
these points. The mission plan consists 
of a series of intermediate waypoints, 
headings to those waypoints, and various 
altitudes to maneuver along the terrain 
following, flying close to the ground 
and terrain avoidance, flying around 
mountains instead of over them, path. 
This plan is used as input to the 
Piloting Expert System for navigation 
during the ingress, at-target, and 
egress phases of a mission. 


3.3 Passive Navigation 

For covert missions, the RAV system 
uses passive ranging of distant ground 
objects to navigate. Passive ranging 
relies only on passive sensors such as 
video and Foward Looking Infra-Red radar 
(FLIR) for its inputs. The Passive 
Navigation System uses these inputs in 
conjunction with data from a digital map 
to calculate algorithmically the current 
aircraft position with respect to the 
terrain. This position is sent to the 
Airspace Expert System where it is then 
distributed to the other RAV systems as 
needed. 


3.4 Airspace Expert System 

The Airspace Expert System (AES) 
provides situational awareness to the 
Piloting Expert System about any 
airspace object whose sphere of 
influence encompasses the aircraft. In 
a three-dimensional airspace, these 
objects can include airports, tactical 
aids to navigation, jet routes, 
circumference of missile sites, and 
other aircraft. 

The AES is a multi-layered system 
with increasing layers of functionality. 
At the core is a spatial database built 
using the TI Relational Table Management 
System. The spatial database contains 
information essential for navigation 
such as aeronautical charts, instrument 
approach plates, and airport directory 

information. Residing on top of the 
spatial database layer is an intelligent 
query translator which provides access 
to information from different database 
sources without the user having to know 
the database structure. A computational 
layer is used to perform basic 
navigational computations such as 
bearing, range, or time to a specified 
point. The highest layer contains 
expert reasoning capabilities used to 
give early warning notifications of 
airspace boundaries or threats and to 
respond to such queries as locate a 
suitable divert base. 


3.5 Piloting Expert System 

The Piloting Expert System (PES) 
has the capability to perform both 
piloting and navigational tasks. The 
PES contains the knowledge to execute 
mission plans which include takeoff, 
standard instrument departure, 
navigation to an IP and target, egress, 
final approach and landing. The PES is 
initially activated by receiving 
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commands from NLMenu or Voice. The PES 
accomplishes a command or series of 
commands by executing the appropriate 
plans in the knowledge base and doing 
any combinations of the following: 
executing the plan from the route 
planner, retrieving spatial knowledge 
from the Airspace Expert System, or 
issuing commands to the Vehicle Control 
Expert System or Vehicle Control System. 

The PES contains multiple knowledge 
bases such as takeoffs, departures, 
navigation, holding patterns, 
approaches, and landings. These 
knowledge bases are written in the TI 
Dallas Inference Engine and Inference 
Corporation’s Automated Reasoning Tool 
(TIDIE/ART) knowledge representation. 
Briefly, the TIDIE/ART representation 
consists of three main components, 
OBJECTS which represent aircraft state 
variables (e.g. airspeed, altitude, 
etc.); NEEDS which designate what task 
is needed (e.g. departure-climb); and 

PLANS which are how that need or 
designated task is to be performed (e.g. 
intercept-inbound-plan) . Within the 
plans are steps which can be either 
event or time driven conditions to be 
met before continuing to the next step 
or actions in the form of directives to 
the Vehicle Control Expert System or the 
Vehicle Control System. 


3.6 Vehicle Control Expert System 

The Vehicle Control Expert System 
( VCES ) has the capability to perform 
basic aircraft and aerobatic maneuvers 
for fighter aircraft. The VCES contains 
the knowledge to perform expert 
autopilot maneuvers such as turns at 
varying bank angles, loops, aileron 
rolls, Immelmanns, and others. Commands 
can be received from NLMenu and Voice to 
execute a maneuver or from PES to 
execute a sequence of maneuvers. 

Also written in the TIDIE/ART 
knowledge representation, the VCES has 
the same structures for decision making 
as the PES but the knowledge is at the 
task level for specific maneuvers as 
opposed to the mission level knowledge 
in the PES. Within a given plan are 
both waits on specified conditions and 
settings of objects such as bank to a 
designated or "target" value. Through 
the setting of various combinations of 
target variables of over 20 different 
objects, the VCES is capable of 
performing any aerobatic maneuver. 


3.7 Vehicle Control System 

The Vehicle Control System (VCS) is 
designed to provide vehicle control of 
the aircraft using the same abstraction 
barrier as a pilot, that of the basic 
aircraft inputs of rudder, stick, and 
throttle. The VCS is responsible for 
achieving and maintaining the target 
values of aircraft state variables set 
by the expert systems. The VCS makes 

changes to the appropriate physical 
control mechanism and monitors the 
progress of the current value toward the 
target value. It continues to make 
corrections as needed until the current 
value equals the target value, or is 
within some tolerance of the target 
value. Then the VCS makes corrections 
that are required to maintain this 
target value. 

The VCS is written in Lisp and runs 
in conjunction with the aircraft 
simulation. The aircraft state 
variables are classified into two 
groups: those whose value is the 
current value set by the simulation 
equations of motion and those whose 
value is the target value set by either 
the VCES or the PES. The variables are 
named by both a particular area such as 
airspeed, altitude, heading, etc. and 
by the physical control mechanism to be 
used such as stick, throttle, 
speedbrake, etc. 


3.8 Intelligent Vehicle Workstation 

The Robotic Air Vehicle program is 
one of the first users of the TI 
Intelligent Vehicle Workstation (IVW), a 
tool for the development of expert 
systems in the area of intelligent 
vehicles. The IVW allows the user to 1) 
define a vehicle platform with equations 
of motion and the environment in which 
the vehiclewill operate, 2) display a 
variety of' viewpoints (e.g. cockpit 
panel, out-of-the-canopy , contour map), 
and 3) observe the vehicle’s behavior 
under the control of the expert system. 
For the RAV system, the IVW houses the 
F-16 equations of motion, displays the 
aircraft dials and gauges, and simulates 
the navigation and communication world. 


3.9 Inter-System Communication 

To communicate between the 
individual systems within the RAV 


337 



system, both a message scheme and 
command interpreter are used. The 
command interpreter translates the 
commands from either NLMenu or Voice 
into the appropriate need for the expert 
systems or into the state variable 
setting for the simulation. The message 
scheme, known as the Postoffice, routes 
commands and other information to the 
proper RAV system. It performs the 
system configuration by installing the 
individual systems on particular 
machines. Thus the Postoffice can 
easily determine the location of the 
system receiving the message and can 
quickly route the message to the correct 
address. 


4 ROBOTIC AIR VEHICLE SYSTEM HARDWARE 
ARCHITECTURE 

The Robotic Air Vehicle system 
software currently resides in a hardware 
configuration of five TI Explorer Lisp 
Machines and one Digital Equipment 
Corporation MicroVax II. This 
conf igurau ion is used during system 
development and test when it is 
advantageous to have the individual 
system's displays visible, however, 
other configurations with fewer 
Explorers can be used. All the machines 
are linked together via Local Area 
Network (LAN). The Explorers, symbolic 
processors, house all the AI software 
and the vehicle control and simulation 
software. The MicroVax II, a floating 
point numeric processor, performs the 
mathematical computations required for 
the conventional software systems. 

4.1 Explorer (TM) Lisp Machine 

The Explorer is a symbolic 
computer, developed by Texas Instruments 
as a tool for developing Artificial 
Intelligence software systems. It is a 
seven slot enclosure, single user 
workstation designed to process high 
level, stack oriented, LISP like 
languages. The Explorer utilizes the 
NuBus (TM) architecture that handles 32 


bit addressing and data. The basic 
Explorer system consists of: a black 
and white monitor with a mouse and 
keyboard, a system chassis, and mass 
storage units. This basic system can 
easily be expanded to include a 
microphone or headset utilized by the 
Voice system, via the connections in the 
monitor. The system chassis contains 
one slot for the addition of optional 
hardware as well as one for additional 
memory, providing up to a total of 24 
megabytes of Random Access Memory (RAM). 
The top surface of the system chassis 
allows the stacking of up to four mass 
storage units. The system hardware has 
built-in self tests and is user 
maintainable, meaning that the user has 
the ability to alter both the system 
hardware and software configurations. 

4.2 Odyssey Processor 

The Odyssey board can be installed 
in an Explorer chassis and connected to 
the NuBus. The processor can 
communicate directly with the RAV 
software environment via its 
Lisp-callable device-service routines. 
Although the Odyssey board is used 
exclusively in the RAV system for voice 
recognition, it also can perform signal 
and image processing for other 
applications. 


4.3 MicroVax II (TM) 

The MicroVax II parallels the computing 
power of the larger Digital Equipment 
Corporation's Vax systems such as the 
11/780, but is smaller in physical 
capacity. The system has 5 megabytes of 
RAM and 70 megabytes of disk storage. 
Since floating point arithmetic is best 
executed on the MicroVax, the 
mathematical computations of the RAV 
route planner and passive navigation are 
performed there and the results 
transmitted across the network to the 
Explorer systems. 
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Figure 2. RAV SYSTEM HARDWARE ARCHITECTURE 
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4.4 Ethernet (TM) And Software 

Protocols 

When more than one Explorer is 
required for software development, a 
means of communication is provided to 
permit asynchronous data transmissions. 
The Explorer is equipped with Ethernet 
hardware and controlling software to 
permit integration into any LAN. The 
Explorers and the MicroVax II used for 
the RAV system are physically connected 
via an Ethernet cable. Ethernet is an 
industry standard for computer 

communications and easily supports a 

multi-system development environment as 
in the RAV system. The network 
communications between the Explorer and 
the MicroVax II is through the 

Department of Defense's protocol 
software, TCP/IP. The Explorer to 

Explorer communications utilize the 
selected standard protocol for the 

Explorer, M.I.T.'s Chaosnet. 


5 RECONNAISSANCE MISSION EXAMPLE 

This section is provided to 
illustrate how the RAV system components 
interact during a typical reconnaissance 
mission. A reconnaissance mission 
consists of the following mission 
phases: takeoff from home base, fly a 
standard instrument departure to an 
initial waypoint, navigate the various 
ingress waypoints to the IP, at the 
target take the recon photos, navigate 
the various egress waypoints, and 
finally perform an approach and landing 
to the home base. 

To start the RAV system, the 
beginning, IP, target, and ending 
waypoints are specified to the route 
planner that produces a terrain 
following/terrain avoidance route. A 
command is issued from either NLMenu or 
Voice to execute the mission from a 
specified airport. The PES processes 


the command performing a takeoff and 
standard instrument departure from the 
specified airport. Once airborne, the 
PES is continually receiving its current 
position from the passive navigation 
system via the AES. Next, the plan from 
the route planner is processed by the 

PES for use in navigating the aircraft 
to the initial waypoint and to each 
successive waypoint. Navigation to 
these waypoints includes the PES' 
issuing of airspeed, altitude, heading, 
radial, and other targets to the VCS. 
To monitor the progress of the RAV along 
the misson, the aircraft's current 
position is dynamically updated on an 
aeronautical section map displayed by 
the AES. A display of the pertainent 
cockpit dials and guages by the IVW 
system continually updates the current 
status of the aircraft. As the RAV 
nears its home base, the PES issues a 
query to the AES to determine the 
bearing and range to the approach fix, 
the point at which final approach 
begins. The AES also updates its 
display to show the approach plates and 
runway diagram of the airport. Using 
the airports Instrument Landing System, 
the PES executes plans for final 
approach and landing. 


6 SUMMARY 

This paper discribed the Robotic 
Air Vehicle system currently under 
development at Texas Instruments for the 
DARPA/AFWAL contract F3 3615-8 2-C-1841 . 
The RAV system has combined multiple 
forms of both numeric and symbolic 
processing to achieve the objectives of 
piloting a robotic air vehicle and 
navigating with passive sensors. 
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Figure 3. RECONNAISSANCE MISSION EXAMPLE 
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